9 research outputs found
Learning to Generate Chairs, Tables and Cars with Convolutional Networks
We train generative 'up-convolutional' neural networks which are able to
generate images of objects given object style, viewpoint, and color. We train
the networks on rendered 3D models of chairs, tables, and cars. Our experiments
show that the networks do not merely learn all images by heart, but rather find
a meaningful representation of 3D models allowing them to assess the similarity
of different models, interpolate between given views to generate the missing
ones, extrapolate views, and invent new objects not present in the training set
by recombining training instances, or even two different object classes.
Moreover, we show that such generative networks can be used to find
correspondences between different objects from the dataset, outperforming
existing approaches on this task.Comment: v4: final PAMI version. New architecture figur
Learning Shape Priors for Single-View 3D Completion and Reconstruction
The problem of single-view 3D shape completion or reconstruction is
challenging, because among the many possible shapes that explain an
observation, most are implausible and do not correspond to natural objects.
Recent research in the field has tackled this problem by exploiting the
expressiveness of deep convolutional networks. In fact, there is another level
of ambiguity that is often overlooked: among plausible shapes, there are still
multiple shapes that fit the 2D image equally well; i.e., the ground truth
shape is non-deterministic given a single-view input. Existing fully supervised
approaches fail to address this issue, and often produce blurry mean shapes
with smooth surfaces but no fine details.
In this paper, we propose ShapeHD, pushing the limit of single-view shape
completion and reconstruction by integrating deep generative models with
adversarially learned shape priors. The learned priors serve as a regularizer,
penalizing the model only if its output is unrealistic, not if it deviates from
the ground truth. Our design thus overcomes both levels of ambiguity
aforementioned. Experiments demonstrate that ShapeHD outperforms state of the
art by a large margin in both shape completion and shape reconstruction on
multiple real datasets.Comment: ECCV 2018. The first two authors contributed equally to this work.
Project page: http://shapehd.csail.mit.edu